Data processing systems such as data storage systems which are coupled to a large amount of data storage may spend a great deal of CPU and I/O BUS time reading and writing data to and from the data storage devices. Reading from and writing data to and from disk drives or similar longer term data storage devices tremendously impacts and decreases performance and throughput of the data processing systems.
Accordingly, many data storage systems now include high speed, short term cache memory which can be accessed much more rapidly by a coupled computer system commonly referred to as a host system without the delays associated with the mechanical motion and time delay inherent in a long term storage device such as a disk drive. Thus, if data requested by a host is stored in cache instead of on the disk drive, system performance can be increased tremendously. Cache memory, however, is a finite resource. It is costly and accordingly, must be properly managed to yield its intended benefit.
Prior art systems or methods aimed at managing cache memory include a system disclosed in U.S. Pat. No. 4,489,378 wherein the system prefetches a selected number of records in anticipation that the data will soon be requested. Several problems, however, exist with such a system. For example, the system has no knowledge of what is already stored in cache and thus, the system must make real time decisions as to whether or not a sequential data access is being performed.
Further, the system prefetches a selected number of data elements without regard for what is currently in cache. Most importantly, such a system is not efficient when used with a multi-tasking/multi-processor system wherein a storage device controller is handling requests in a multiplexed fashion from many hosts. In such a case, multiplexed sequential data access would appear as a non-sequential or random data requests to the cache manager and the desired data would not be present in the cache due to the non-sequential appearance of the task.
An additional prior art system for controlling or managing cache memory is disclosed in U.S. Pat. No. 4,853,846 wherein a cache memory directory is split up into as many segments as there are processors or hosts which are accessing the memory. Given the associated high cost of cache memory, such a system results in expensive, duplicative cache memories. In addition, such a system is also incapable of handling multi-tasking systems given that the system requires a dedicated cache directory associated with each accessing host or processor.
Further, both referenced prior art systems as well as other similar prior art systems are not capable of allowing the user to establish selectable thresholds or criteria to determine the threshold for determining that a sequential task is indeed in progress, the number of data records to prefetch once a sequential task has been identified, and the amount of cache memory to be allocated to a given task.
Cache memory is also utilized when data is being written from a host computer to a long term data storage device such as a disk drive. In such systems, data may be written to short term, high speed, cache memory in which it is temporarily held with an indication that this data must be written to longer term data storage when the data storage system finds it convenient or otherwise has the time to perform this operation. When utilizing short term, high speed memory, such as cache, to temporarily hold write pending data, memory storage locations are removed from the main pool of memory storage locations generally available to the data storage system in which data may be held pending use by the host.
Accordingly, when data which is "write pending" has been written to a longer term storage media, the memory locations containing the now written data must be returned to the main memory pool as quickly as possible in order to ensure that enough memory remains available for data which is being transferred from the longer term data storage device, such as a disk drive, to the memory, awaiting retrieval by a host system.
Further, due to the random usage of cache memory, cache memory addresses containing data which have recently been written to longer term storage media must be returned to the main memory pool in such a manner as to ensure that if the data which has recently been written to longer term storage media will not be utilized, those cache memory locations should be made available as quickly as possible to the main cache pool. If, however, it is possible that the data having just previously been written to longer term storage media may be utilized by the host, then this data should be retained in memory for a period of time sufficient to ensure its accessibility to the host.
Prior cache management or data processing systems, however, have not utilized methods to control usability of short term, cache memory which contains data to be written or which has recently been written, to longer term data storage media. An additional difficulty exists in those systems which attempt to anticipate data requests by identifying sequential data access "jobs" by looking only for a predetermined number of sequential tracks in cache memory using only the virtual address of the track. Even a pure random data access will cause the data storage system to access data with adjacent addresses in a random manner over a long period of time, thus making the cache management system falsely believe that a sequential data access is in progress when in fact there is not. This false indication that a sequential data access is in progress will cause the cache management system to prefetch additional unnecessary data into the cache which will not be utilized but merely take up valuable and finite cache memory space.
A further difficulty which exists in the prior art systems, is ensuring that any one process running on and accessing the data storage system does not utilize the entire cache. For example, a process which is copying one 4,500 track storage device to another storage device could potentially use up 4,500 locations of cache. Such very high cache memory usage would greatly stress the cache and leave no cache available for other jobs or processes. Such a stressed cache condition must be immediately discovered and the cache usage limited to a small number of cache locations since, in such situations, the data is not normally utilized more than one time.
Accordingly, what is needed is a system and method for managing short term memory storage locations which contain data to be written to long term data storage media in such a way as to ensure that the memory addresses are made available to a main memory pool as quickly as possible, without sacrificing or overwriting data which resides in high speed, shorter term memory which may be utilized by a coupled host computer in a relatively short period of time.